LB 3045 
F7 
liCopy 1 




Book___Q__ 
CopightE". 



CjDEXRIGHT 0£POSm 



\ 



TEXTBOOK SELECTION 



TEXTBOOK SELECTION 

-V BY 
R. H. FRANZEN 

DIRECTOR OF RESEARCH, DE8 MOINES PUBLIC SCHOOLS 

AND 

F. B. KNIGHT 

STATE UNIVERSITT OF IOWA 

With an Introduction by 
ERNEST HORN 




BALTIMORE, MD. 

WARWICK & YORK, Inc. 

1922 






Copyright, 1922, by 
Warwick & York, Inc. 



©CI.A686812 



THE MAPLE PRESS - YORK PA 



OCT 10 '^2 



INTRODUCTION 

It is becoming increasingly clear in the minds of 
those who face the practical problems of improving 
instruction, that text books usually determine the 
success or failure of any educational method. And 
yet the various problems involved in text book making 
have been attacked mainly through the hazardous 
and wasteful method of trial and error, or by the 
method of rough judgments. Recently, however, 
an inclination to regard the selection of text books 
as a matter for scientific study has become prevalent. 
Every study dealing with students or social values or 
economy in learning has been promptly seized upon 
by authors or publishers either to guide the construc- 
tion of new books or to justify books already 
published. 

Studies of social utility in the various school sub- 
jects have been particularly effective in changing the 
content of text books. This is easily seen in the 
reconstruction of books in spelling, and in the reor- 
ganization of the texts in arithmetic. It is now 
practically impossible to sell spelling books which do 

5 



6 TEXTBOOK SELECTION 

not at least claim to be based upon the recent scientific 
studies of the vocabulary of correspondence. 

There is also rapid response to varying types of 
educational theory. Both the subject matter and 
the organization of books are effected as the propa- 
ganda for new theories becomes wide spread. How- 
ever, both publishers and superintendents are coming 
more and more to distrust mere philosophizing as a 
method of determining the organization of school 
books; the publisher because these philosophical 
theories are often transitory; the superintendents 
because books based upon these theories are often 
found to be unsatisfactory. It is becoming increas- 
ingly clear that all theories for the construction of 
text books must line up with the criteria now available 
from the scientific appraisals of social needs, and with 
the criteria available from the experimental studies in 
the economy of learning. 

These studies in the economy of learning, including 
the study of performance tests, are of the deepest 
significance. Some data are now available in almost 
every field and kind of learning. They range all the 
way from those learning the more elusive problems 
of determining tastes and interests to the somewhat 
simpler problems involved in determining practice 
exercises for specific performances such as hand 
writing. The two studies reported in this book are 
excellent examples of the utilization of scientific 



TEXTBOOK SELECTION 7 

techniques in determining student reactions to text 
book materials. 

At a time when a new emphasis upon motivation 
is being made, Dr. Knight's study of students' choices 
in Uterature is very timely. We have only begun to 
appreciate the importance of utilizing the method of 
judgments of choices under properly controlled condi- 
tions. Certain aspects of learning are particularly in 
need^of just such appraisals. It is fortunate to have 
an example which does not end merely as a statement 
of a general principle but in a helpful, concrete 
recommendation which can be immediately utilized 
by all teachers of high school English. 

It was inevitable from the first that the various 
studies of economy in silent reading would react upon 
the construction of text books. Dr. Franzen's 
investigation was made in the practical emergency of 
text book selection. It illustrates what may be 
expected more and more in the way of substituting 
data for opinion in the solution of the problems 
involved in text book adoptions. There is reason to 
believe that in the next few years we may expect 
much experimental activity along these lines. 

Both of the studies suggest the interesting possibil- 
ity of subjecting manuscripts to some such experi- 
mentation before printing. It would seem that authors 
owe this to the publishers, and both authors and 
publishers owe this to the public. Of course, scientific 



8 TEXTBOOK SELECTION 

studies take time and are expensive. It is not unrea- 
sonable to expect that the profession of teaching is 
now on such a basis that the rewards for taking such 
pains will make the research worth while. 

Ernest Horn. 



CONTENTS 



Page 

Introduction 5 



CHAPTER I 

The Cbiteria for the Selection op Textbooks ... 11 
The Factor of Interest. 
The Factor of Comprehension. 

The Permanent Methods of Study Involved in the Text. 
The Permanent Value of the Content. 
The Mechanical Construction of the Text. 
The Fundamental Proposition of Textbook Selection. 
Data vs. Opinion. 
Best Opinion vs. Poor Opinion. 
Pooled Judgment. 
Neglect Irrelevancies. 

CHAPTER II 

Study One: Applying the Criterion op Interest to 

High School English Literature Texts ... 22 
Classification of Books in Order of Interest. 
The Readers' Interest is the Criterion. 
The Literary Merit Tests of the Lists. 
Description of Actual Data. 

The Reliability of the Quality and Quantity of the 
Data. 

9 



10 CONTENTS 

CHAPTER III 

Page 

Study Two; Applying the Criterion op Comprehen- 
sion TO Geography Texts for Elementary 
Schools 66 

The Accumulation of the Data. 

Selecting Texts to Meet the Needs of Particular Grades. 

The Success of Texts in Reaching Their Reading Public. 

Individual Differences in Needs of Various Grades. 



CHAPTER ONE 

PRINCIPLES OF TEXT SELECTION 

The major criteria for the selection of text-books 
are five: 

(a) The factor of interest. 

(6) The factor of comprehension. 

(c) The permanent methods of study involved in 
the text. 

(d) The permanent value of the content. 

(e) The mechanical construction of the text. 

These criteria possess determinative importance in 
the opinion of educators. There is no one way of 
expressing these criteria. Methods of statement differ 
but the meat of the matter is constant. We find 
the above statement convenient for our exposition 
but claim no particular rhetorical superiority for it. 

At present, it is difl&cult to apply these criteria in 
a particular situation. The application of general 
principles to concrete situations has always been 
troublesome . Educators find themselves in no uncom- 
mon impasse when they attempt to apply readily 
accepted principles to the specific problems of their 

11 



12 TEXTBOOK SELECTION 

profession. The purpose of this nionograph is a 
double one; first to state the fundamental propositions 
which must underlie sound practice of text-book 
construction; second to give two researches which are 
at least moderately successful attempts at such appli- 
cation. The researches are (a) the application of the 
broad doctrine of interest to the problem of selecting 
texts for high school English Literature work, (6) 
the application of the factor of comprehension to the 
selection of geography texts for elementary schools. 

The history of public education in the United States 
is very far from a monotonous report of blunders and 
inadequacies. A study of text-book construction and 
of the methods of text-book selection even now preva- 
lent, however, gives the student of education reason 
to prophesy much increase in the genuine adequacy of 
public school text-books. Further we are sure that 
the selection of a particular text in preference to other 
available books is about to become far less a matter of 
guess and poorly based opinion and far more a matter 
of deliberate choice in the light of sound criteria of 
text-book excellence. 

Text-book selection is obviously a process of making 
choices, and choices can be made in many ways. It 
is quite possible that certain texts have been chosen 
not because they were better tools of instruction than 
other books, but because the salesman urging their 
adoption had a more persuasive vocabulary, more 



TEXTBOOK SELECTION 13 

agile and plastic sales methods, or was politically 
more canny than his competitors. Other texts have 
had large sales because they were or professed to be 
exponents of some pedagogical doctrine which momen- 
tarily hypnotized the buying agent. Better than the 
above, many texts have been adopted by schools 
because the proper authority, having studied the 
matter deliberately, chose those particular books. 

Fundamental Peopositions 

The thoughtful school administrator will pre- 
sumably subscribe to the following four propositions 
as fundamental to sound practice in text-book selection. 

I. Irrelevant differences pertaining to texts should 
be utterly neglected, viz. pressure from salesmen, 
exhilarating sales tactics. Texts should be chosen in 
light of the five determinative criteria of good text 
construction, namely: 

(a) The factor of interest. 

(6) The factor of comprehension. 

(c) The permanent methods of study involved in 
the text. 

(d) The permanent value of the content. 

(e) The mechanical construction of the text. 

II. Objective evidence should be given preference 
over subjective opinion wherever objective evidence 
can be reasonably obtained. 



14 TEXTBOOK SELECTION 

III. Wherever subjective opinion is resorted to the 
final judgment should not be the result of the wisdom 
of any one person. Final judgment should be the 
product of the pooled votes of several competent 
judges. Among these judges should be at least on^ 
class room teacher who is to use the book. The more 
mutually independent the judgments are the more 
stable the final result. 

IV. Wherever subject opinion has to be substituted 
for objective measurement, especial care should be 
taken to get the best opinion possible. A very good 
judge may sometimes give a very hasty opinion. 

Objective Evidence 

Proposition I, using relevant criteria, and proposition 
III, using the insight of more than one judge, need no 
further discussion. The main part of this book is an 
attempt to apply proposition II to actual, concrete 
school situations. Objective evidence is usually 
better than subjective opinion. Many of the decep- 
tions of opinion have been pointed out of late. But 
the objective evidence has its weak points also. Not 
all evidence is good evidence. Mere quantity of 
evidence does not involve in a mysterious way a 
guarantee that the evidence is reliable or consistent 
or determinative. Objectivity of evidence never 
compensates for the errors which come from wrong 



/ 



TEXTBOOK SELECTION 15 

methods of gathering the evidence or mistakes in the 
interpretation of the ever so objective data. 

It is hoped that the two problems of text-book 
selection dealt with in this book will be helpful not 
only because of the particular findings but also because 
of the exposition and discussion of the methods used 
to gain objective evidence concerning texts. 

The methods used are of general application. 
Those who are responsible for text-book selection 
should know the reading diflSculties not only of geog- 
raphies, but also of all texts, especially arithmetic, 
histories, and science books. 

One of the problems is the determination of the 
reading difficulty of geographies. Obviously a child 
with fifth grade reading ability will work to dis- 
advantage if he uses a text which assumes a reading 
ability rarely found below the seventh grade. 

The fourth fundamental proposition of text-book 
selection is "Where subjective opinion has to be 
used, care must be taken to obtain the best opinion 
possible. '^ This means the abandoning of general 
opinions and the persistent analysis and estimate 
of the specifics of each text judged. 

We no longer rate a school building as a whole. 
We judge it for ventilation, for play space, for class 
rooms, for many elements which taken all together 
make a school house. So with text-books, we should 
make our judgments not in general, but in particular. 



16 / TEXTBOOK SELECTION 

It is apparent that score cards for text-books are useful 
instruments to aid in making analyzed judgments. 
Unfortunately such score cards are not in general use. 
In fact for many texts they are not in existence. 

The Use of Rating Score Cards 

It is clear that the comparison of one text-book with 
another will be complete when both texts have been 
measured in the light of the five criteria. The whole 
rating of text-books can finally be expressed in mathe- 
matical terms on -a, score card. For purposes of 
making a decision as to which of several texts to 
adopt the value of each text can, theoretically, be 
expressed in one figure. The figure would be the sum 
of the scores given to each of the five criteria if each 
criterion were considered as important as every other 
criterion. Weighted scores would be needed if the 
criteria were of unequal value. That these five 
criteria are of unequal value seems to be the case. 
From a pool of many judgments we assign following 
point values: 

Interest 200 

Comprehension 250 

Permanent value of subject matter . 250 

Value of method 200 

Mechanical elements . 100 

Total 1000 



TEXTBOOK SELECTION 17 

Score cards or rating schemes of one kind or another 
are generally used to get in orderly fashion expert 
opinion.^ Many decisions in practical school affairs 
will have to be based on opinion rather than other 
data for some time to come. Wherever this is the 
case it becomes exceedingly important to obtain the 
best opinion in the best way possible. By best 
opinion we mean the judgment of those not only who 
know most about the matter but the opinion of 
enlightened judges who take the trouble to make the 
best judgment they can make. - Here is where the 
score card is of value. Opinion expressed through 
score card ratings will not give insight the judge 
himself does not possess, but opinion expressed 
through score card ratings has a better chance of 
being opinion up to the limit of insight than unanalysed 
opinion has. Thus we see that the worth of opinion 
varies in merit not only with the true wisdom of the 
judge but, also, is a function of the method of obtain- 
ing the opinion. Analysed opinion through score 
card methods increases the chances for a judge to get 
all his goods to market. The better the score card 
the better the judgment will be. A Biology Score 
Card, one of a series, is given here. Its use makes 
judgment better because it forces the judge to con- 

1 Opinion of any degree of expertness should be resorted to 
only in the absence of evidential data. 



18 TEXTBOOK SELECTION 

sider many aspects of his problem he might otherwise 
neglect. An even better score card would make for 
even better judgments. 



TEXTBOOK SELECTION 



19 



Tentative Score Card for Judging Value op Biology 
Texts 



Name of^book. 

Author 

Publisher 

To be used in . 
Scored by 



I. Interest 

(If there is experimental evidence neg- 
lect all sub-topics) 

A. Mention of animals and plants 

known by student 

1. Wild in this district 

2. Domestic in this district 

B. Possibility of demonstrations which 
emphasize value of this study to the 
student 

1. In the laboratory 

2. Outside 

a. Back yard biology 

b. On exciirsions 

C. Style 

1. Vividness 

2. Clarity 

D. Correlation with other subjects. . . . 

1. Other sciences 

2. Literature 

3. History 

II. Comprehension 

(If there is experimental evidence neg- 
lect all subtopics) 

A. Reading difficulty 

1. Vocabulary 

2. Style 



200 



250 



50 



50 



50 



50 



150 



20 



TEXTBOOK SELECTION 



Tentative Score Card for Judging Value op Biology 
Texts {Continued) 





I 


A 


I 


a 


B, DiflBculty of concepts 


250 
200 




50 

50 

90 
80 

80 

\ 
150 




25 
25 

50 
30 

40 

40 
75 




20 

10 
10 








2. Degree of involved presentation. 

C. Difficulty of graphs and illustrative 

material 




III. Permanent Value of Subject Matter . . . 
(If there is, experimental evidence 




A. Accuracy and'balance qi represen- 
tation of the subsidiary sciences 

B. Development of scientific interests . 
1. Sympathetic but critical atti- 




2. Stimulation to further work% . . . 

C. Dynamic concept of evolution 

1. Material on progress and evolu- 
tion. 

a. Development from lower to 
higher forms in plant and 




b. Development of nervous sys- 
tem, use of hands and use 
of voice; and their values 




c. Differences of animals defined 

as differences of their wants 

as well as their abilities. . . 

2. Possibilities for development in 




IV. Value of method 




A. Organization of material 




1. Opportunity to develop g;eneral 
principles 









TEXTBOOK SELECTION 



21 



Tentative Score Card for Judging Value of Biology 
Texts (Continued) 



a. Sufficient data to make induc- 

tion possible 

b. Summaries 

c. Indices 

d. Illustrations and maps 

e. Graphs 

2. Opportunity for application of 

general principles 

a. Suggestions for outside work . . . 

b. Classroom exercises 

B.' Correlation with other subject mat- 
ter 

1. Stimulus to more reading 

2. Relation to other work 

Mechanical elements 

(Use objective measurements and norms 

to judge these values wherever possi- 
ble) 

A. Size and clearness of print of text. . . 

B. Distinctness of pictures 

C. Size and clearness of print of foot 

notes 

D. Size and clearness of print of mar- 

ginal notes and indices 

E. Width of margins 

F. Length of Unes 

G. Paper 

H. Binding 

I. Size and shape of book 



100 



50 



75 



CHAPTER II 

APPLYING THE CRITERION OF INTEREST TO 

HIGH SCHOOL ENGLISH LITERATURE 

TEXTS 

Section One 

The business of this study is to do two things: 

A. First.— To name and classify in order of inter- 
est the best liked books commonly used in high 
school hterature work. This list will be referred to 
as the Preferred List. 

Name in order of interest the next best liked books 
commonly used. This list is referred to as the Second 
Preferred List. 

Name in order of interest the third best liked 
books. This list will be called the Substitute List. 

Name in order of violent disUke books commonly 
used in high schools. This list will be referred to as 
the Free Reading List. 

B. The second section of our business is to (1) Defend 
the proposition that the Preferred List of books should 
form the basis or core of the English literature work 
in the high school, that books in the Second Pre- 
ferred List should be used next frequently, that the 
Substitute List may be drawn from after the Pre- 

22 



TEXTBOOK SELECTION 23 

ferred and Second Preferred are exhausted and that 
books in the Free Reading List should never be used.^ 

This may be restated as follows : The best available 
criterion to use in selecting English literature books 
is the amount of interest students have in the books, 
considering only those books which are now in 
common use. 

(2) Present and interpret data calculated to establish 
the validity of our lists. 

The Pkeferred List 

In the second section of this study sufficiently valid 
data are presented to warrant the assertion that the 
following are the best liked books commonly used in 
high schools of lowa.^ 

Careful students who wish to inquire into the factual 
basis of our lists before reading the lists will do better 
if they read the second section first. The busy 

* Books in the Free Reading List should never be used as a 
class text. These books should be in the library for the use of 
those very few students who wish to read them. 

^ Our data, we hasten to point out, are based on generous 
sampUngs from high schools of Iowa. Our criterion of interest 
would apply strictly only to Iowa. The lists of books are gener- 
ally appUcable in so far as any large group of students have 
about the same interests as large groups of Iowa students. 
Some exceptions in interest are probably important. For 
instance Uncle Tom's Cabin is highly interesting to Iowa high 
school pupils. Its interest rating would probably be much 
lower in Georgia or Louisiana. 



24 TEXTBOOK SELECTION 

school administrator can assume the vaHdity of the 
evidence and content himself with a perusal of the 
lists. 

Table A. — Preferred List 

Reliability 
Interest of 
Name of Book Rating^ Rating^ 

Laddie 97 1.4 

Vicar of Wakefield 92 1.7 

Kenilworth 79 3.7 

Treasure Island 66 2.5 

Hamlet 63 2.5 

Comus 61 3.5 

Uncle Tom's Cabin 61 3.0 

Wild Animals I Have Known 60 5.1 

BenHur ...59 3.0 

Last Days of Pompeii 59 4.1 

The Spy. 59 3.6 

Romona 58 4.4 

King Lear 57 2.2 

The Crossing 55 4.3 

Snowbound 54 2.5 

Luck of Roaring Camp 54 4.0 

Macbeth 53 2.2 

Ivanhoe 53 3.8 

Carlyle's Essay on Burns 53 3.3 

1 Interest rating is in terms of the percentage of students, 
who, having read the book, declare that it was of distinct 



TEXTBOOK SELECTION 25 

Second Preferred List 

Incidents of a French Camp 53 4.2 

The Virginian 52 3.8 

Les Miserables 52 4.5 

The Crisis 51 3.1 

The Last of the Mohicans 51 2.6 

Up From Slavery 51 5.5 

Tom Sawyer 50 2.3 

Tale of Two Cities 50 3.0 

Silas Marner 49 2.2 

Man Without a Country 49 2.4 

.Huckleberry Finn 49 2.7 

Washington's Farewell Address 49 3.3 

Ancient Mariner 48 2.2 

The Gold Bug 48 2.5 

Richard Carvel 48 5.3 

Miles Standish 47 3.2 

Robinson Crusoe 46 2.6 

Lorna Doone 46 4.0 

interest to them. Thus the higher the rating, the better the 
book according to our criterion. 

2 Reliability of Rating is to be read as follows : If many 
more estimates of the book were taken, the interest rating might 
go up or down as much as the reHability figure. The chances 
of its going up or down by an amount greater than the reported 
unreUability are very few. Thus The Vicar of Wakefield has an 
interest rating of 92 and an unreliability of 1.7, which means 
that as much more data are obtained, the 92 may go up to 93.7 
or down to 90.3. The technique of ascertaining the reHability 
is fully dealt with in section two. 



26 TEXTBOOK SELECTION 

Free Reading List 
In Order of Demerit 

Contains books Never to be Used in High School 

Paradise Lost 6 1.4 

Lay of the Last Minstrel 10 2.5 

Heroes and Hero Worship 11 2.8 

Essay on Warren Hastings 12 3.4 

Macauley's Essay on Johnson 12 2.6 

Autocrat of the Breakfast Table 13 2.0 

Vision of Sir Launfal 13 1.2 

Childe Harold 18 3.5 

Webster's Bunker Hill Speech 19 2.8 

Fairie Queene 20 2.9 

Essays of EHa 21 4.5 

Lays of Ancient Rome 21 4.4 

Utopia 23 4.5 

Thoreau's Walden 23 4.4 

Gareth and Lynette 23 2.9 

The Princess 24 3.5 

Tales of a Wayside Inn 24 2.7 

House of Seven Gables 24 2.1 

Pilgrim's Progress 25 2.5 



TEXTBOOK SELECTION 27 

Substitute List 

Contains in order of probable interest to students 
books less interesting than the First and Second Pre- 
ferred List, but of measurably greater interest than the 
books of the Free Reading List. The books are listed 
in order of interest merit. 

Twelfth Night 46 4.9 

Launcelot and Elaine 45 2.6 

Hoosier Schoolmaster 45 3.5 

Christmas Carol 44 2.5 

Hans Brinker 44 3.5 

Legend of Sleepy Hollow 43 2.1 

David Copperfield 43 2.7 

Joan of Arc 43 3.9 

Enoch Arden 43 2.8 

Lincoln's Gettysburg Address 42 2.2 

Leatherstocking Tales 42 3.6 

Merchant of Venice 41 2.0 

Burke's Conciliation 41 2.9 

Midsummer-Night's Dream 40 3.6 

Prisoner of Chillon 40 3.1 



28 TEXTBOOK SELECTION 

The interest ratings of 27 other books frequently 
read in high schools have been determined. The 
First, Second Preferred and Substitute Lists contain 
those of highest ratings. These lists contain more 
books than would ordinarily be used in a four-year 
high school. It seemed needless to print the others. 
These interest ratings may be obtained on request. 

Notes on the Lists 

The criterion used in setting up an order of merit 
of English literature texts is pupils' interest in the 
text. This proposition is defended in Section II. 
Any one whose choice of texts in the practical situa- 
tion of actual use in high schools is influenced by 
these lists should bear the following in mind : 

(a) The interest which students have in the texts 
is not the only criterion to be used in deciding which 
one of two possible books to use. Other things being 
at all equal, however, it is better psychology and 
sounder common sense to prefer interesting material 
over material of no interest or positive dislike. 

(b) Not every book that could be used in high 
school is included in this study. No book is dealt with 
here unless it has been reported by at least eighty 
of the high school graduates who rated the list. Thus 
we are concerned only with books which are already 
fairly commonly accepted as worth while. This 



TEXTBOOK SELECTION 29 

matter could be stated: English literature pro- 
fessors and teachers have on the whole approved of 
many books for high school work. More books are 
generally approved than can possibly be read by any 
one student. Of books accepted as useful which shall 
be chosen? Those of greatest interest to the pupils 
is our thesis. 

(c) It is not certain that the lists are equally sound 
for all sections of the country. In some instances we 
know radical variations in interest would be very 
probable. Children's interests must vary with the 
traditions and interests of the section in which they 
live; how much is not known. Probably books 
vary in their relative universality of appeal. Thus 
Shakespeare may be of about the same interest 
strength everywhere, whereas Uncle Tom's Cabin may 
have high interest in some sections and practically 
none in others. We are well within the facts on the 
other hand to suspect the absence of radical and sharp 
changes in interest. The children of one county 
have about the same interests as those in a contiguous 
county. There is no apparent reason that on a basis 
of children's interests there should be endless variation 
within the same state. It might be added that there 
is probably less difference in interests in city versus 
rural youths than is ordinarily assumed. To differ- 
entiate books on a basis of sex is idle, for boys and 
girls have to read the same books in most high schools. 



30 TEXTBOOK SELECTION 

There is no reason why our lists might not very closely 
approximate at least mid- western lists. 

(d) Other criteria should be used in making up a 
list for a particular high school. It is sound to hold 
that other things being equal books in the Preferred 
List should be read sometime during high school. The 
books in excess of this Preferred List should be chosen 
from the Second Preferred List. However, a book 
with an interest rating of 54 % poorly taught will prob- 
ably do less service than a book with an interest rating 
of 48% well taught. Thus, while no book should be 
used from the Free Reading List, in a particular situa- 
tion should a teacher much prefer a 48% book to a 
54% book it is good sense to let her teach it, for the 
differential of the teacher's attitude is quite probably 
more important than a small difference in percentage 
of interest. 

(e) Should the preponent elements in the situation 
be a set of slightly lower interest books, on hand, and 
in good condition, a set of slightly higher interest books 
available only through purchase, a supply of funds 
so limited that the purchase of the new books means 
going without some other good thing; then to delay 
for a time the desired replacement is probably entirely 
defensible. 

(/) The interest which one has in any book is due 
not only to the character of the book but also to the 
character of the reader and the method of reading. 



TEXTBOOK SELECTION 31 

Thus the same book though read under equivalent 
conditions will not have the same amount of interest 
for all students, not because the book varies but 
because students differ greatly in what they want out 
of a book. Differences may lie in many fields such 
as home training, type of play indulged in, amount of 
travel, ability to read, type of fantasies or day dreams 
enjoyed, etc. Thus, even the most interesting books 
on the list can not be expected to interest all pupils. 
We know they will not. We are pretty sure the books 
in the Preferred List will very likely be actually dis- 
liked by some students. In addition a book which is 
generally preferred to a second one may in some 
peculiar instances be less liked by a class or school 
than the second. However the theory of proba- 
bility as used in this study is a sound technique in 
curriculum construction. 

Exceptions need not greatly disturb us. Obviously 
a book in and of itself may be inherently more 
interesting to pupils than some other. But if the 
other book happens to be well taught and the book 
inherently better happens to be grossly mistaught, 
then the inferior book may very likely appear to be 
better. 

(g) A certain variety in texts is undoubtedly 
desirable. Had the Preferred List on a basis of inter- 
est been all drama, or all poetry, or all heroic novel, 
practical consideration would force us to draw heavily 



32 TEXTBOOK SELECTION 

on other school literature. But the Preferred List as a 
matter of fact is pretty well balanced as far as kinds 
of literary content are concerned. And many wide 
deviations from this list can be comfortably accounted 
for by the consideration that while what large groups 
tend to like, other large groups do not tend to dislike, 
in individual cases no one would expect all pupils to 
like the same books and in the same relative order of 
interest. These lists do not pretend to be correct 
for all students, but as far as we now know they will 
strongly tend to be better than any other list. 

(h) It is not useful to think of the most interesting 
book as the "best" book or the least interesting book 
as the "worst" book. The most interesting book is 
Laddie. No one would claim that Laddie is the best 
book in the sense that "if only one book was to be 
read in high school this book should be read." Such 
choices and such "ifs" do not have to be made. 
Many books are read. The question is which group is 
best. In the best group Laddie finds a place and 
Paradise Lost does not. This placing is, of course, 
according to our criterion of interest. 



TEXTBOOK SELECTION 33 

Readers' Interest the Criterion^ 

The criterion used in the construction of these lists 
is the interest of the student as reported. 

No one aware of the administrative exigencies of a 
high school would contend that ''what pupils like" 
should be the sole criterion for admitting a book, a 
subject, a technique, or a rule into the total economy 
of high school procedure. Neither would one suspect 
that what pupils dislike should be the sole criterion 
for dropping a book, a subject, a technique, or a rule 
from the school. 

It is hardly conceivable that any teacher is so 
unenlightened as to hold that disagreeable tasks are 
worth while because of their disagreeableness. A 
task may be distasteful and yet worth while, but its 
worth-while-ness is certainly due to other factors than 
its unpleasing nature. 

Theoretically, most competent English teachers 
would agree that one of the criteria safely used to 
select texts would be students' interest, but there 
would be little agreement on the amount of influence 
interest should have. If the books available should 

1 Theoretical consideration of the importance of interest is 
assumed. Personallj'', we view those who do not appreciate 
the pragmatic importance of interest in education as in need of 
instruction in the simple principles of the teaching process. 

Nor are we concerned here with the objectives of our English 
Literature work. 

3 



34 TEXTBOOK SELECTION 

be scored on a basis of 100 points, would literary 
merit be assigned 50 points, moral teaching 25, and 
interest 25, or would it be reading difficulty of the 
composition 50, literary merit 15, interest 15, amount 
book is used in other schools 20, — or what relative 
importance would be assigned to interest? 

In our present ignorance of the details of scientific 
management in education, one would utterly despair 
of getting definite answers to questions of this kind. 
However answers to such questions must be obtained 
before we can speak of education as a science in the 
sense that civil engineering is a science. All would 
agree that interest is only one of several factors which 
properly influence selection. How influential a factor 
it should be has not yet been scientifically established. 

Our data show that the selection of English literature 
in the State of Iowa practically refuses to recognize the 
factor of interest at all. The frequency with which 
books are read in Iowa high schools and the frequency 
with which students report these books as interesting 
to them have practically no correlation other than 
that of chance. This roughly may be taken to mean 
that the more frequently read books are rated as less 
than average interest just as often as they are rated 
as above average interest, also that the less frequently 
read books receive an interest rating greater than the 
average as often as "less than the average" rating. 
Such a state of affairs would be justifiable if it could 



TEXTBOOK SELECTION 35 

be clearly shown that other criteria more important 
than students' interest determined the frequency with 
which books are read. That such criteria do not 
operate will be shown and evidence given later in this 
study. 

Assuming that the 783 reports on which this study 
is based are a fair sampling, the following examples of 
the strained relations if not absolute divorce between 
*'how often books are read" and ''how much books 
are liked" will be illuminating. 

The 19 books read with the greatest interest were 
recorded on the average 280 times. 

The 20 books read with enough interest to warrant 
being in a Second Preferred or First Substitute Lists 
were read on the average 247 times. 

The 20 books read with so little interest that they 
should not be read at all were read on the average 
203 times. 

The average interest rating of the Preferred List 
is 62.7%. 

The average interest rating of the Free reading List 
is 16.3%. 

Thus, while in frequency the shrinkage in use 
between the best and worst lists is only 17.6%, the 
shrinkage in interest is 46.4%! The shrinkage in 
interest is approximately 2.6 times as great as the 
shrinkage in frequency of use between the Preferred 
and the Free Reading List. I doubt if any one would 



36 TEXTBOOK SELECTION 

want to defend this relationship. It certainly would 
never have happened if the choosers of text-books 
had known the interest in books before making their 
choices. 

To those who may suspect that a study of only 
the best books and the worst in point of interest hides 
a more favorable state of affairs because it does not 
consider the middle books at all, the relation between 
frequency of use and amount of interest of all books 
will be given. 

The technique here is that of correlation. 

We digress to explain what is meant by correlation. 
Persons familiar with statistical theory should skip the 
next page. 

The coefficient of correlation is a measure of con- 
comitant variation between two series of facts about 
the same data. The limits of this coefficient are 
— 1 and +1. If the relationship is + 1 it means that 
there is a perfect matching. Thus if there were a -f-1 
correlation between the frequency with which the 
books we are studying are read and the amount of 
interests pupils have in them, then the following 
conditions would hold: 

The book read most frequently would also be the 
most interesting; the book read least frequently would 
also be the least interesting; the book read with the 
fifth greatest frequency would also be the fifth book in 
interest; the book read next to the least often would 



TEXTBOOK SELECTION 37 

be the book next to the bottom in interest, and 
so on. 

If there were a —1 correlation between frequency 
and interest then just the reverse would be true, i.e., 
the most read book would be the least interesting; the 
book fifth from the top in frequency of the reading 
would be fifth from the bottom in its interest rating. 

If the correlation were zero then there would be 
no relationship but a chance one between frequency 
and interest. Some much read books would be high 
in interest, others low, others average. Some books 
low in frequency would be low in interest, others high, 
others average in interest. Thus the sign and size of 
the correlation coefficient measures the amount of 
mutual relationship between the frequency with which 
books in our list are read and the amount of interest 
pupils have in these same books. ^ 

Using only the books which were reported eighty 
times or more (the unreliability of ratings based on less 
than eighty gets too large for conservative study) we 
find the correlation between frequency of use and 
interest to be — .04, practically zero. 

1 Further explanation of the correlation coefficient may be 
found in 

Thorndike. Mental and Social Measurements. 

Brown and Thomson. Essentials of Mental Measurement. 

Rugg. Statistical Method Applied to Education. 

Refer to index in each book. 



38 TEXTBOOK SELECTION 

This means that there is no tendency to read the 
more interesting books oftener than the non-interest- 
ing one. There is also no tendency to read the non- 
interesting ones any more frequently than the 
interesting ones. 

In view of the fact that the data give also a rating of 
positive dislike it is instructive to study the correlation 
between the frequency with which books are read 
and the positive dislike for them. It should be 
pointed out that the positive dislike ratings are 
entirely separate from any interest ratings. They 
were made independently. The correlation is —.52 
between frequency of reading and positive dislike. 
This means that there is a pretty definite factor of 
negative selection going on at present. The nearest 
approach to anything like scientific control that 
the data reveal is this fact that on the whole books 
which are positively disliked tend to be read less often 
and there is a slight tendency to read them less often 
in the order of the strenuousness of dislike . 

Certainly the factor of interest should operate in the 
selection of texts. But it does not. We are not 
prepared to say that it is theoretically impossible to do 
good work in a subject in which the learner has no 
interest, but interest is a factor which facilitates 
learning. On the whole, interest in the work runs 
along with skill in it. With material such as litera- 
ture the interest factors are probably more important 



TEXTBOOK SELECTION 39 

than in more mechanical subjects where physical 
skill based on habit predominates. If reading books 
carries no interest but often even positive annoyance, 
there is little if any likelihood that reading goes on 
except under compulsion. tThe frequent task of 
reading inherently uninteresting material under school 
compulsion may account for whatever truth there 
is in the general feehng that adults on the whole 
read little literature of great merit^"^ Very few high 
school graduates ever read the Mneid over again for 
the fun of it. We doubt if one high school graduate in 
fifty, who, having read Sesame and Lilies or Paradise 
Lost in high school, ever reads these books again, or 
reads other works by the same authors unless they are 
in turn teaching them in high school for a wage, to 
pupils who will never look at them or think of them 
again unless later as high school teachers, they in 
turn — and so on. 1 

Several cross sections of the data are presented here 
to further illustrate the divorce between interest and 
frequency of reading certain books. 

Table B contains the 29 books read the most fre- 
quently. On these we have 350 reports or more. 

* The way a book is taught must determine in part the inter- 
est pupils have in it. This relation between how a book is 
taught and resulting interest is as yet unknown. 



Table B 

No. OF 

Times Place on Interest 

Reported Name of Book Rating^ 

720 Vision of Sir Launf al Free Reading List 

607 Merchant of Venice Substitute 

552 Legend of Sleepy Hollow. . . . Substitute 

535 Lady of the Lake. No place 

529 Silas Marner Second Preferred 

523 Macbeth First Preferred 

523 Lincoln's Gettysburg Speech Substitute 

503 Ancient Mariner Second Preferred 

492 Ivanhoe First Preferred 

481 Courtship of Miles Standish. Second Preferred 

465 Great Stone Face No place 

460 The Raven No place 

459 Julius Caesar No place 

424 Man Without a Country Second Preferred 

414 Gold Bug Second Preferred 

411 House of Seven Gables Free Reading List 

397 Christmas Carol Substitute 

385 Last of the Mohicans Second Preferred 

383 Canterbury Tales No place 

383 Snowbound First Preferred 

380 Passing of Arthur No place 

378 Sketch Book No place 

375 Hamlet First Preferred 

369 Treasure Island First Preferred 

368 Thanatopsis No place 

368 Idylls of the King No place 

366 Robinson Crusoe Second Preferred 

366 Gray's Elegy No place 

357 Launcelot and Elaine Substitute 

i"No place" means that the book has an interest rating 
better than the Free Reading List but not high enough to reach 
even our Substitute List. 



TEXTBOOK SELECTION 41 

Of the 29 most frequently read books (350 reports 
or more out of 783 students) 

5 are in our First Preferred List 
7 are in our Second Preferred List 
5 are in our Substitute List 
10 are in ''Not Placed" list 
2 are in the Free Reading List 
Total 29 

Of the ten books ^which had an interest rating 
of 45, one was read 640 times and one but 160 times. 
The other eight were rather evenly distributed 
between these extremes. Of the twelve books read 
with an equal frequency (280 times) one had an inter- 
est rating of 10, another of 95. And here the other 
ten were scattered between these extremes. 

All the data show how books with equal interest 
are read with highly variable frequency. Books read 
with equal frequency have highly variable interest 
value. 

So far we have shown that interest as such exerts 
no measurable influence on frequency of reading over 
the state of Iowa as a whole. We wish to present 
data to show that the operation of other definite 
criteria is not evident. 

If clean cut criteria were operating under conscious 
direction some one would know about it. Who does? 
What are the criteria? Reading difficulty is a very 
good criterion, but the reading difliculty of books 



42 TEXTBOOK SELECTION 

used in high school English literature has never been 
determined. Some one could very well determine this. 
Perhaps information value is thought by some to 
be an effective criterion which we have carelessly over- 
looked. We have yet to see a study of the information 
value of the books under consideration. Such a 
study might well be made. At present, however, this 
criterion is still a ''poor relative" of opinion and a 
very much be-fussed relative into the bargain. It 
would be difficult to show that the much read books 
are higher in information value than the much liked 
books. Comparing the First Preferred List with the 
Free Reading List one might perhaps be impressed 
with the realism of the Preferred List in contrast to 
the fantasy of the Free Reading List. Possibly 
people have to be older than of high school age before 
they become sufficiently disillusioned about the world 
to find pleasure in fantasy. The high school pupil 
may be still satisfied with the world as it is, or nearly 
is. He may not yet be interested in Utopias, Para- 
dises Lost or Found, Fairie Queens or Visions, or he 
may be too old to be interested in them. 

LITERARY MERIT NO CRITERION 

It is always possible to fall back on the well known 
defense of literary merit. It could be asserted, if 
one is careless enough about the facts, that the 
criterion of literary merit decides the frequency with 
which books on the whole are read in high school. 



TEXTBOOK SELECTION 43 

In one sense this is true. Probably no book is much 
read until by a vague consensus of opinion it is worth 
reading. But this does not effect our data because 
we have considered only books rather frequently 
read. They all possess enough literary merit to be 
generally used. But differences in literary merit in 
excess of a minimum amount of it to allow entrance 
of books into common use can not account for the 
frequencies. 

We have accepted literary merit in this sense, as 
a working criterion of a minimum amount. It is 
important to realize that in no other sense can we hold 
it to be influential. It is not at present a criterion for 
two reasons. First, there is no objective measure to 
use as a criterion. There may be some day a scale for 
rating for literary merit. Up to now contributions on 
the matter have been much cluttered with opinions 
and not disciplined with scientific or orderly study. 
In short, every one has a literary rating all his own, 
(which rating we suspect is largely a rationalization of 
likes and dislikes by the individual doing the rating) . 

Second. — If differences in literary merit decided 
differences of frequencies of use so that the most meri- 
torious book was read the most frequently, the second 
most meritorious book was read the next most fre- 
quently, and so on, then there would have to be a 
general agreement as to the differences in literary merit 
of the books concerned. There is not even an approxi- 
mation of agreement, as the following data show: 



44 TEXTBOOK SELECTION 

We have had five groups of judges rate the nineteen 
books composing the Preferred List and the nineteen 
books composing the Free Reading List for the quahty, 
literary merit. The judges were in all instances igno- 
rant of the interest value of these books. They were 
urged to give their own opinion independent (as far as 
possible) of fame of the author, popular appeal of the 
book, etc. 

The value A was to be given to those books which 
were of preeminent literary value, the great books 
of the ages. 

The value E was to be given to those books which 
were of such poor quality that probably they should 
not be considered for serious work at all. 

The value C was to be given to books about half way 
between A and E. 

The value B was to be assigned to books better 
than C but of less merit than A. 

The value D was to be assigned to books worse than 
C but better than E. 

The additional instruction was given that in case 
of unfamiliarity with a book no rating need be given. 

The pooled results of these judgments of the literary 
value of the Preferred List (each book given a number) 
and of the Free Reading List (each book given a 
number) is presented. The reader is asked to dis- 
tinguish on a basis of literary merit which list of 
books is the best. 



TEXTBOOK SELECTION 45 







List X^ 










Showing the distribution of judgments on 


nineteen 


books for literary merit 


A, B, 


C, D, 


E, amounts of 


merit 


as definec 


in text 


. Numbers 1, 


2,3, 


etc. 


refer 


to books. 
















A 


B 


C 


D 


E 




T 


1. 


3 


21 


27 


6 


3 




60 


2. 


9 


41 


30 


6 


1 




87 


3. 


21 


27 


7 


2 







57 


4. 


16 


56 


43 


4 







119 


5. 


52 


46 


16 


11 


2 




127 


6. 


3 


34 


22 


10 


1 




70 


7. 


25 


30 


38 


22 


2 




117 


8. 


24 


32 


10 


15 


10 




91 


9. 


51 


31 


36 


"2 


2 




122 


10. 


14 


28 


49 


6 


7 




104 


11. 


9 


33 


38 


13 


3 




96 


12. 


22 


26 


39 


13 


4 




104 


13. 


48 


31 


26 


5 


1 




111 


14. 


14 


22 


21 


20 


4 




81 


15. 


30 


33 


47 


7 


1 




118 


16. 


9 


31 


20 


12 


3 




75 


17. 


53 


37 


16 


6 


1 




113 


18. 


18 


51 


29 


4 


1 




103 


19. 


7 


17 


21 


12 


6 




63 



T 428 627 535 175 52 1817 

1 Whether this list is the Preferred or the Free Reading List 
will be stated later on. This postponement is to enable the 
reader to see the utter hopelessness of getting a "literary 
merit" when the books are given by number rather than name. 



46 TEXTBOOK SELECTION 

List Y 





A 


B 


C 


D 


E 


T 


1. 


27 


44 


18 


1 


3 


93 


2. 


3 


19 


27 


2 





51 


3. 


25 


29 


24 


4 


3 


85 


4. 


6 


2 


9 


16 


11 


44 


5. 


9 


36 


33 


4 


3 


85 


6. 


51 


25 


18 


3 





97 


7. 


21 


52 


20 


3 


1 


97 


8. 


3 


27 


31 


7 


4 


72 


9. 


3 


31 


35 


4 





73 


10. 


9 


29 


23 


7 


3 


71 


11. 


8 


6 


19 


3 


3 


39 


12. 


10 


22 


8 


4 


3 


47 


13. 


36 


21 


13 


14 


3 


87 


14. 


18 


31 


15 


3 


1 


68 


15. 


1 


25 


23 


2 


2 


53 


16. 


15 


35 


24 


2 


3 


79 


17. 


39 


26 


35 


5 


1 


106 


18. 


12 


17 


44 


7 


6 


86 


19. 


30 


37 


31 


5 


2 


105 


T 


326 


514 


450 


96 


52 


1438 



TEXTBOOK SELECTION 47 

The reader will readily see that a great amount 
of sagacity would be necessary to determine whether 
List X or List Y is superior in literary merit. There is 
certainly not enough difference to outweigh the essen- 
tial difference in interest value which exists between 
the two lists. 

The careful student will immediately wonder if our 
pooled ratings are not a mixture of inexpert and con- 
sequently valueless opinion. We shall therefore spread 
out the pith of each group of ratings to show that 
there is no difference of importance between the 
judgments of groups which vary in expertness. 

Group I is a class in psychology of advertising. 
This group is not expert in matters of literary merit of 
English literature. Their judgments are; 

List X List Y 

of 369 judgments of 230 judgments 

on 19 texts on 19 texts 

21.4% judgments were A 12.2% were A 

35.6% judgments were B 38.7% were B 

31.5% judgments were C 42.2% were C 

9.5% judgments were D 4.3% were D 

1.9% judgments were E 2.6% were E 

We interpret these data as showing that in the 
opinion of a psychology of advertising class List X 
is slightly superior to List Y in literary merit. 



48 



TEXTBOOK SELECTION 



Group B is a class of college juniors preparing to 
teach in high schools. We assume that their judg- 
ments are slightly more expert than the judgments of 
Group A. 



List X 

of 995 judgments 
on 19 texts 

28.3% of judgments were A 

36.8% of judgments were B 

24.7% of judgments were C 

6.6% of judgments were D 

3.5% of judgments were E 



List Y 

of 803 judgments 
on 19 texts 

19.9% were A 

39.9% were B 

29.0% were C 

8.5% were D 

3.6% were E 



In the judgment of Group B List X appears to be 
slightly better than List Y. The chief characteristic 
of this difference is however its negligible smallness. 
Other criteria than literary merit should obviously 
operate in deciding which list to use. 

Group G is a graduate seminar in educational 
psychology. Most of the group are experienced 
teachers. They are presumably still more expert 
than either Group A or B. 



TEXTBOOK SELECTION 49 

List X List Y 

of 216 judgments of 222 judgments 

on 19 texts on 19 texts 

26.3% of judgments were A 36.5% were A 

35.6% of judgments were B 31.1% were B 

27.3% of judgments were C 23.9% were C 

9.3% of judgments were D 6.3% were D 

1.4% of judgments were E 2.3% were E 

Here List X has 61.9% of judgments B and A 
List Y has 67.6% of judgments B and A. 

There is a slight difference in total literary merit in 
favor of List Y. But the difference is so slight that no 
one could possibly judge it to be as important psycho- 
logically, pedagogically, or culturally from the teach- 
ing standpoint as is the difference in interest value 
determined by this study. 

Group D, our most expert group, is Doctor 
Craig's seminar in introduction to graduate work in 
English. Many of the group are instructors or have 
been instructors in English literature. They have 
profited by unquestioned expertness in English 
literature through graduate study in English. 



50 TEXTBOOK SELECTION 

List X List Y 

of 182 judgments of 176 judgments 

on 19 texts on 19 texts 

22.5% of judgments were A 14.2% were A 

21.9% of judgments were B 27.8% were B 

34.6% of judgments were C 38.6% were C 

15.9% of judgments were D 14.7% were D 

4.8% of judgments were E 4.5% were E 

Two facts are apparent: (1) the expert group rated 
both lists on the whole lower than the relatively less 
expert groups rated them; (2) in the judgment of the 
most expert group List X and List Y are practically of 
equal literary merit. Perhaps List X is slightly 
better. In all cases List X is our Preferred List and Y 
our Free Reading List. 

The force of the argument is now apparently 
adequate. 

When groups of judges varying in expertness are 
asked to pass on two lists of books the experts hand in 
about the same findings as the less expert. There is 
then no expert opinion as different from common 
knowledge. For if there were a useful expertness 
graduate students in literature would have it. They 
do not. Our blunt conclusion is: since judgments of 
literary merit on our Preferred List as a whole and on 
our Free Reading List as a whole can not be made to 
even suggest significant differences, then literary 



TEXTBOOK SELECTION 51 

merit as a criterion, except as we have already admitted 
it^ is not determinative. This we take it leaves a clear 
field for the validity and usefulness of the lists pre- 
sented in the first of this study which are based on the 
criterion of interest. ^ 

This lack of agreement in matters of merit is not 
surprising. After all literary merit or any of its 
synonyms contains but little useful truth. It is 
similar to such phrases as the Well-rounded Life, 
Education for Complete Living, True Culture, Good 
Citizenship, and the like. We assume that any phrase 
such as those contains a notion upon which many can 
agree as they can agree on the notion that two plus 
two equals four. No such notion is contained by such 
a phrase as Literary Merit, or Education for Complete 
Living, or Good Citizenship. These are verbal fog 
screens concealing chiefly differences of opinion. 
After all what may appear to you as a well-rounded life 

1 Minimum amount to be admitted at all. 

2 If the reader should feel that there are a few books in the 
Free Reading List of such great merit that they should be 
read in spite of their low interest value he can satisfy himself 
as to the difference of opinion on the matter by attempting to 
pick out from the pooled judgments of List Y which books they 
are by the spread of judgments presented there. He might 
possibly pick number 6, 13, 17, which are Autocrat of the 
Breakfast Table, Utopia, Tales of Wayside Inn. No one of 
these books are really liked by one pupil out of four who have 
read them. 



52 TEXTBOOK SELECTION 

may appear to your neighbor as the Hfe of a prig, a 
selfish hoarder, a silly person, or what not. While his 
notion of a well-rounded life may seem to you to be a 
life too physical, too intellectual, too ethereal, or too 
worldly. And all the while we know you are both 
wrong and that our notion of the well-rounded life 
though sharply differing from both of yours is really 
correct. 

So with literary merit, perhaps that is meritorious 
which is liked or which satisfies, or in some other way 
leaves a characteristic impression on the reader. As 
you and your neighbor are forever satisfied with dif- 
ferent things, literary merit is one thing to you and 
another' to him. 

Even though this explanation is faulty the cold 
fact remains that there is as yet too little agreement 
as to the varying worth of the books we are considering 
to use merit as a criterion. 

Third, — if this were at present a criterion, it cer- 
tainly is not effective. For who would say the follow- 
ing differences in frequency of reading were due to 
clear differences in literary value : 



TEXTBOOK SELECTION 53 

Vision of Sir 

Launf al read 720 times out of 783 reports 

King Lear read 185 times out of 783 reports 

Merchant of Venice, read 607 times out of 783 reports 

Hamlet read 375 times out of 783 reports 

Julius Caesar read 459 times out of 783 reports 

Treasure Island. . . .read 369 times out of 783 reports 
Sesame and Lilies. . .read 145 times out of 783 reports 
Gold Bug read 414 times out of 783 reports 

Another defense for the present situation would be 
that a balanced literary ration must be provided for 
high school pupils and the present practices are 
approximations of this balance. The necessity for 
balance is of course entirely proper and sound. But 
there is some difficulty in justifying the wide spread 
notions about just what a balanced ration is. How- 
ever, it is to be noted that there is at least variety in 
the Preferred List as well as in the other lists. There 
is a fairly good variety both as to the type of literature, 
date of publications {i.e., age) and many of the books 
in our lists have undisputed literary merit. We 
doubt if in the matter of balance the books most often 
read make any better presentation than do the books 
which when read are of distinct interest. 

The present situation in English literature as far 
as the frequency with which books are read is con- 
cerned, is certainly not the result of conscious, 
purposeful, scientific control and direction, right or 



54 TEXTBOOK SELECTION 

wrong. It is a result of a confusion of influence hope- 
lessly interwoven. In addition it should be kept 
clearly in mind that even if other criteria were oper- 
ating still the present impotence of the interest factor 
can be viewed with philosophic calm only by the 
pedant.^ 

The evidence so far supports the following 
statements : 

The correlation between the criterion of interest 
and the frequency of reading is zero. 

The criterion of reading difficulty can not operate 
in a useful fashion for the difference in reading diffi- 
culty has never been determined. 

The criterion of useful information is not operative 
for the useful information rating of the books is too 
nebulous. 

Literary merit doubtless operates in the restricted 
sense of determining in some way the entrance of 
a book into use at all. But it does not determine the 
relative importance of books that are used, for there 

1 It can be shown statistically that the operation of definite 
criteria need not be assumed to account for the present situa- 
tion. For the frequency with which books are read conform 
to the upper half of a pure chance curve within the probable 
error. The treatment of the data to prove this is deleted for we 
do not need it to prove our point concerning the absence of 
steering and the prevalence of drift at present. Moreover our 
interest is not to prove things are wrong but to sell better 
English literature choices to school managers. 



TEXTBOOK SELECTION 55 

is not sufficient agreement as to the measuring of 
literary merit. 

We repeat that the use of some criterion though 
imperfect is better than the vague and uncertain 
practices which now exist. If choices of high school 
literature texts were made in the light of the lists 
of this study the situation would not be made worse 
and would probably be genuinely improved. The 
criterion of interest is not only known but is of 
respectable importance in any teaching situation. 

Section II 

We have presented lists of literature texts. 

We have defended the use of the criterion of interest 
in setting up lists. 

However, the lists are good only to the extent 
that the data upon which they are based are valid. 
We now present a description of the original data. 

The freshmen English classes at the State University 
of Iowa were given as an exercise the filling out of the 
following blank. 1 

We took freshman data not because those only were avail- 
able, but because freshmen appeared to us to be the proper 
judges. High school seniors are a Httle too near some books to 

^ The data were obtained through the English Department 
of the State University of Iowa of which Professor H. C. Craig 
is the head. For this courtesy we desire to express our appre- 
ciation. 



56 TEXTBOOK SELECTION 

have a good perspective. College seniors' ratings would have 
been the result of original preference plus four years of college 
training which would certainly tend to change their attitude 
toward some books. Seniors would tend to rate not on what 
they did like but on what they now think they ought to like. 
There would be a repetition of the "old oaken bucket" delu- 
sion. The necessity of carrying water for mother, the weight 
of well sweep, the leaks in the bucket, the spilled water, the ice 
on the well, all tend to be forgotten as distance softens the hard 
spots and enchants the pleasantness of childhood. I suspect 
that many honest grown-ups really think that in their youth 
they said at story time " Mother, tell us a story and be sure to 
put in a wholesome moral." 



Questionnaire 

1. Check in the first column those books which you studied in 
class during your high school course. 

2. Check in the second column those books which you read out 
of class as collateral reading. 

3. Check in the third column those books which you now believe 
to have been of the most interest. 

4. Check in the fourth column those books which you now 
believe to have been of no particular interest, and yet not 
entirely lacking of interest. 

5. Check in the fifth column those books which you now believe 
to have been of no interest whatever, and worthless. 

6. Check in the sixth column those books which you would drop 
from a course. 

7. At the bottom of the last page you will find space. List here 
a number of books you would add to a course. 

Name 

High school last attended 

Major study in S. U. I 

Age 



Burke's Conciliation of the Colonies. 

McCaulay's Essay on Ben Johnson. 

Macbeth. 

King Lear. 

Midsummer-Night's Dream. 

TweHth Night. 

Hamlet. 

The Ancient Mariner. 

Silas Marner. 

Ivanhoe. 

Vicar of Wakefield. 

Snowbound. 

Roger De Coverly Papers. 

Irving's Sketch Book. 

Great Stone Face. 

The Last of the Mohicans. 

The Vision of Sir Launfal. 

The Courtship of Miles Standish. 

etc. 



58 TEXTBOOK SELECTION 

This list contains 108 books. 

Returns were received from 732 students. These 
data were tabulated in ways to permit of convenient 
study. 

The correlations which are reported in this study 
were computed by the Pearson formula 

There are, of course, certain objections which can 
be raised about the trustworthiness of the data. I 
shall consider first the trustworthiness of the data as 
to the quantity, then as to the quality of it. 

Quantity of the Data 

The ratings of the books in this study can be 
accepted with confidence as far as the amount of evid- 
ence is concerned. We shall discuss at some length 
the reliability of the ratings because they are expressed 
in percentages. The reliability of percentages has so 
far been too little considered in scientific studies in 
education. 732 questionnaires were returned. Not 
all the books were read by every one of the 732 
students. In fact, no book was reported on by all. 
Obviously the more cases there are in any study the 
greater the reliability of the findings, other things 

^ See Thorndike Mental and Social Measurements, page 173. 



TEXTBOOK SELECTION 59 

being at all equal. We could have wished to base our 
interest ratings on 7320 cases instead of 732. Our 
evidence is much stronger because it is based on 732 
reports than it would have been if only 73 students 
had rated the books. 

Since the reliability of the ratings is a function 
of the number of reports studied we want to know 
just how reliable our findings are. To illustrate: 
Hamlet was judged by 375 students. 63% judged 
the book to be of genuine interest to them. Can we 
generalize from this and say 63 % of high school pupils 
taken at large find Hamlet of distinct appeal to them? 
Of course the ideal way to determine what percentage 
of high school pupils find Hamlet of especial fasci- 
nation would be to ask all of those who had read 
Hamlet. 

Less ideal than this way but far better than guessing 
is the determination of the reliability of the 63%. 
This can be done by the application of the formula of 
simple sampling. Each one of the 375 students, who 
having read Hamlet, rated it, can be thought of for 
our purposes as performing a little experiment. He 
is to rate Hamlet as interesting or not interesting. 
375 rate Hamlet and 63% of them rate it as 
interesting. Suppose we went on and got 700 ratings. 
Would 63% of the 700 rate as did the first 375? The 
percentage of 63 would probably vary slightly. But 
variations would probably center around 63. After 



60 TEXTBOOK SELECTION 

700 or 1,700 more students had been asked we should 
expect their percentage of interest to be nearer 63 
than 12 or 15 or even 50 or 90. That is, on our 375 
cases we should expect that the interest ratings of a 
much larger group would not be exactly 63 but that 
63 would be the best single bet we could make in the 
light of our facts. 

The formula y/npq when n = number of cases, p=% 
of pupils who rate Hamlet as interesting, and q the % 
who rated the book as not interesting, will tell us the 
standard measure of variation from 63 which the 
percentage of interest will take as more cases are 
added. 

To illustrate the process in detail: 

n = 375 students rated Hamlet. 

p = 63 % of these rated Hamlet as interesting. 

q = 37% of these did not rate Hamlet as interesting. 

With 375 cases to predict from how will the per- 
centage of interest probably vary as many more cases 
are added? 

This is found by applying formula * 

71 



A/ 7Vt)0 

In our illustration = 2.55%. 



1 The quantity -^wpq is itself divided by n to change it from 
a quantity in terms of n to a percentage. 



TEXTBOOK SELECTION 61 

This means that as more cases are added 63% may 
go up 2.55% or down 2.55%. The probabiHty that 
the % of interest will not go over 65.55% (63 + 2.55) 
or below 61.45% (63 - 2.55) are 68 chances out of 
100. The probability of the interest in Hamlet being 
less than 57 or more than 69 is but two chances out of 
100. It is evident that, when only those cases where 
n is 80 or more are involved, the reliability of our 
findings is very high indeed. So much for the 
validity of our data as to quantity.^ 

It is possible that we have enough data but its 
quality is such that it is untrustworthy. Here the 
argument would run as follows. If you ask any num- 
ber of people a question about which they know 
nothing the answers are no better than guesses. The 
average of many fake answers is not a right answer 
except by chance. 

College freshmen may not know why they liked 
and disliked certain books in high school, but it is 
reasonable to assume the plain facts that pupils do 
like and dislike books and that they know what 
interests them and what does not. If their reports 
are false they are false not because knowledge was 



1 Those who wish to further satisfy themselves on either the 
soundness of the theory of "Sampling" or on the technique 
involved will find Yule's Introduction to the Theory of Statistics, 
Chapter XIII, a satisfactory reference. 



62 TEXTBOOK SELECTION 

lacking but because they purposely lied or reported 
with hopeless carelessness. 

There was certainly no reason for deception. There 
may have been a slight tendency to say the proper 
thing which in this case would be to report more 
favorably than their true interests warrant on books 
which are famous, or old, or "high brow" and to 
underrate their true interest in the so-called popular 
books. The fact that the percentages of positively 
disliked books are relatively small may perhaps be 
interpreted as a result of two factors: — genuine 
dislike minus a hesitation of going on record as 
disliking books that one "ought'' to like. To the 
extent that our data are vitiated by this hesitation, 
it gives a case weaker not stronger than the true 
case in favor of our Preferred List and against the 
Free Reading List. 

The following relationships of the data are our 
evidence of the validity of the data. 

The correlation between interest and a neutral 
attitude toward the books was — .18. 

The correlation between neutral attitude and dislike 
was +.51. 

The correlation between dislikes and dislike so 
great that dropping the book in question was recom- 
mended was +.67. 

These three correlations go right against any notion 
that the ratings were the product of carelessness. In 



TEXTBOOK SELECTION 63 

fact, if the markings were the product of carelessness 
rather than of thought we simply could not account 
for these correlations. 

The correlation between interest and a neutral 
attitude ( — .18) is practically no relationship at all 
other than a chance one. And it is about what one 
could reasonably expect to be the fact. The great 
interest that some have in a book can very well run 
along with a neutral, 50-50, attitude of others. 
Unless one is under the spell of the 'Apathetic fallacy'' 
he would not expect to be able to establish any high 
correlation between the indifference of one group and 
the interests of another. 

He would however expect more alienation to exist 
between the interests of some high school pupils and 
the positive dislikes of others than between the inter- 
est of some high school pupils and the neutral attitude 
of others. There is nothing in the nature of children 
or in education which creates a tendency of likes of 
some to correlate with the dislikes of others more 
closely than with a neutral attitude. Just the oppo- 
site would be more likely. This is exactly what 
we get. 

There is a negative correlation of signilScant size 
between likes and dislikes ( — .377); there is none 
between likes and neutral attitudes. This inner 
consistency of the data argues potently for the validity 
of the data. If likes and dislikes had correlated posi- 



64 TEXTBOOK SELECTION 

tively instead of negatively it would cast a severe 
criticism against the quality of the data. They 
correlate negatively. 

The correlation between neutral attitudes and dis- 
likes was +.510. This shows that there is some 
positive relationship between these two series of 
ratings. What some rate as of indifferent value, 
others tend to rate as disliked. A common sense 
interpretation of this might be that in expressing 
opinions, indifference of some predicts dislike of others. 
Perhaps timid souls mark as indifferent what more 
valiant spirits rate as disliked.^ 

One other correlation further leads us to maintain 
that our data were honestly given and represent the 
facts — the correlation between dislike and great dis- 
like, i.e., enough to advise dropping from the course, 
is +.67. This can only be interpreted if we accept 
the validity of our data. Surely there ought to be a 
high positive correlation between a tendency to dislike 
and a tendency to dislike a great deal. This relation- 
ship ought to be greatest of any in the data. It is 
the greatest relationship. 

Every correlation we have fits in with a notion that 
the pupils gave a fairly true report of their attitude 

* If one wished to make a general observation of human 
nature concerning the reports of people it would be : indifference 
is more closely related to dislike than to positive like or interest 
as we use the term. 



TEXTBOOK SELECTION 65 

about books read. // their ratings had been false or 
overcareless no such consistency of relationship would 
have been at all probable. 

The data upon which the lists are constructed have 
a good rehability for two reasons. 

(a) The quantity of the data is too great to admit 
of a distorted picture due to inadequate sampHng. 

(6) The inner consistency of the relationship 
between 

Interest and a neutral attitude (= —.184) 
Interest and dislike ( = — .377) 

Neutral attitude and dislike (= +.510) 
Dislike and great dislike (= +.670) 

gives us plenty of evidence to accept the qualitative 
superiority of the data. All the original data and 
subsequent computations used in this study are on 
file in the College of Education, State University of 
Iowa, Iowa City, Iowa. 



CHAPTER III 

APPLYING THE CRITERION OF COMPRE- 
HENSION TO GEOGRAPHY TEXTS FOR 
ELEMENTARY SCHOOLS 

Comprehension Values of Geography Texts 

All the tables used in this study are put together 
at the end. It is believed that this will facilitate the 
study of this research. 

This report, as part of the judgment of relative 
value of geographies, deals with the comparative 
difficulty which children find in reading the various 
texts understandingly. The method used in this 
study is of general use. No fundamental change in 
it is necessary were we dealing with history or science 
or other texts. We would define the comprehension 
value of a text as the degree to which children may 
read passages of that text and answer simple questions 
on such excerpts without entering into any intricate 
consideration of the difficulties involved in compre- 
hending the reading matter. We can then arrive at a 
very adequate notion of the quality which we are 
comparing. With such a formulation in mind, we 
have formed random samplings of the reading matter 
of each text book into tests and have asked simple 

66 



TEXTBOOK SELECTION 67 

questions on the basis of each sampHng. We will then 
judge the value of any one book by children's responses 
to questions on its subject matter. 

There were five series of books which were candi- 
dates for selection. The series were lettered and each 
first book was called ''1/' and each second book was 
called "2.'^ The names of the five series are deleted 
for obvious reasons. Throughout this study they 
will be referred to as series A, B, C, D, and E. 

One could not hold that reading difficulty or ease should alone 
decide which text to use. The other criteria of selection must 
be given due weight. Books of reading difficulty too far above 
or below the reading ability of the pupils who use the books are 
undesirable. Quite often superior subject matter necessarily 
involves slightly greater reading difficulty. For instance, a 
superficial description of the "tides" may involve easier reading 
than an adequate discussion of the ocean movements. Should 
the more difficult book be used the teacher should know where 
the hard passages are and govern herself accordingly. 

In order to ascribe to the book itself the adequacy 
or inadequacy of the responses of children to ques- 
tions asked, after the reading of any portions selected 
from it, we need to be perfectly sure of three major 
conditions that may influence the behavior, aside from 
the actual difiiculty of the material in the text. We 
must know: 

1. That the samplings are a fair representation 
of the book and that the questions on the passages 



68 TEXTBOOK SELECTION 

from one book are no more difficult than the questions 
on passages form any other book. 

2. That the children who answer questions on one 
book are just as capable as those who answer questions 
on any other book. 

3. That the order in which the tests are given has 
no effect on the responses of the children who take 
them. 

The data which follow are organized to present 
results which adequately account for these three 
factors, and consequently we may attribute the values, 
as indicated through the responses of the children 
directly to the texts. 

The Faibness of the Sampling 

1. If the responses of the children to the questions 
on any one book were materially affected by chance 
inclusions of difficult passages, there would be no 
relation between the scores made by a group on the 
first halves of the tests and the scores made by the 
same group on the second halves of the tests. A group 
might then get a high percentage of the questions 
forming the first half of the selections correct, and a 
low percentage on the second half of those selections 
correct. On the other hand, if on the whole the books 
which are acclaimed as easy by selections forming the 
first half of the test (which are taken from the first 



TEXTBOOK SELECTION 69 

half of the text) are also acclaimed as easy by selec- 
tions forming the second half of the test (which are 
taken from the second half of the text), then we can 
safely decide that the difficulty of the selections is a 
function of the book. Also such agreement would 
indicate that the general difficulty of the questions 
has not been greatly influenced by the mood or ability 
of the author of the test. Agreement between the 
average difficulty of the first half with the average 
difficulty of the second half of the test, shows that the 
author's personality was not a determining factor in 
this difficulty. 

It could be maintained that all of the questions 
to one book were easy, whereas all of the questions to 
another book were hard, but we do not believe that 
this is a serious possibility because of the method used 
in construction of the tests. The tests were, of course, 
constructed with strict impartiality — passages being 
taken from the same pages of each book, and crucial 
questions being asked from each passage, with no 
consideration except the logic of the passage, as the 
author understood it. 

Table 1 gives the agreement as it is. It will be noted 
that the books which are easy to read, by the verdict 
of children's responses to the first halves are also easy 
to read by such verdict of the second halves. Those 
that are hard to read when we consider the first half 
are hard to read when we consider the last half. 



70 TEXTBOOK SELECTION 

There is only one exception, and that is D-1 and D-2. 
The first halves of these two books are easy to read, 
and the second halves are hard to read. It is impor- 
tant to note that these two exceptions are of the same 
series. Those books were written in that way. 

The correlation between values by the first half 
and values by the second half is + .32. The correlation 
of values by the first half with values by the total is 
+ .76. The correlation of values by the second half 
with values by the total is +.83. Though correla- 
tions with cases as few as ten are unallowable, when 
the object is to show relationship between qualities 
or between tests, it is a valuable means of showing 
characteristics of actual data, no matter what the 
number of cases. It would be unjustifiable to say 
that first halves of tests such as these correlated as 
a rule with second halves about .32, but it is entirely 
justifiable to say that these actual first halves do 
actually correlate .32 with these actual second halves. 
By Brown's formula between the totals with totals 
obtained in the same way it goes up to .48. 

It will be profitable to note in this connection that 
each second book of a series is harder than its first, 
and that the volumes I place the series about as the 
volumes II do, except the E-1 and E-2, and that of all 
the series, it would be most likely that the difficulty of 
the first volume of the E series had a different relation 
to the other first volumes than the second volume of 



TEXTBOOK SELECTION 71 

this series had to the other second volumes, since 
Mr. X. wrote the first volume and Mr. Y. wrote the 
second, whereas in all of the other series, volumes 
I and volumes II were both written by one or by both 
men. 

The Equality of the Groups Used in the 

Tests 

2. Children whose responses are being evaluated to 
gain comparative estimates of the values of these 
texts are all twelve-year-olds. Since they are all of the 
same age and since there are from about 350 to about 
500 in each group, the abilities of the groups must be 
about equal, provided that their verbal intelligence is 
about equal. Table 2 gives the distributions on Wylie 
Opposites test A, B, and C, given to these same 
children about one year before the Geography Read- 
ing tests were given. These children were then 
eleven years old, and Table 2 represents the distribu- 
tions of Wylie scores for eleven-year-olds of the same 
schools that housed the twelve-year-olds who took 
the geography tests. ''A" represents distributions of 
Wylie scores of eleven-year-olds January 14, 1921, 
in the same schools where the twelve-year-olds took 
the Geography Reading tests, A-1 and A-2, early in 
1922. '^B'* represents distributions of Wylie scores 
for eleven-year-olds January 14, 1921, in the schools 



72 TEXTBOOK SELECTION 

where the twelve-year-olds took the Geography- 
Reading tests, B-1 and B-2, early in 1922, etc. 
Table 3 gives these same facts for twelve-year-olds, 
January 14, 1921, by groups of schools which took the 
various Geography Reading tests. Table 3 gives the 
facts which insure us that it is not characteristic of 
any of these groups of schools to have twelve-year-olds 
which are much brighter than eleven-year-olds, and 
that therefore the relative abilities of the twelve-year- 
olds in these groups of schools will be just about what 
those same children were one year before, when they 
were eleven-year-olds. 

In Table 2 the percentage of zeros was for the 
'^A'^ group, .050; for the ^'B" group, .070; for 
the "C" group, .070; for the ''D" group, .067; for 
the "'E" group, .058. The lines punctuating the distri- 
butions are percentile lines. The first line in each 
distribution marks off the lowest 10% from the other 
90%. The next line marks off the lowest 25% from 
the highest 75%. The third line marks the lowest 
50% from the highest 50%. The fourth line marks 
the lowest 75 % from the highest 25 %. The last line 
marks the lowest 90% from the highest 10%. The 
remarkable correspondence of the position of these 
lines in the five groups in both Table 2 and Table 3 
shows beyond any doubt that the groups of children 
are identical, so far as ability on the Wylie test is 
concerned. 



TEXTBOOK SELECTION 73 

Negligible Effect of the Order of Giving the 
Tests 

3. The possibility that the order in which tests were 
given might affect the responses of the children and 
would therefore introduce a factor in the evaluation 
which could not be justly attributed to the book itself, 
needs consideration. It has been repeatedly found 
that tests constructed as these were, are not easily 
amenable to practice. We found at Garden City that 
giving practice in this function so as to make an 
appreciable difference in the average of the groups 
took two years of special training; and Dr. Arthur 
Gates found at Scarborough, where the abilities were 
already near the intelligence limit, that practice in the 
function made no difference at all in the test. 

All the ''2's'' were given directly after the ''I's," 
and still each ''2'' is harder than its corresponding 
''!." This seems to indicate that the order of giving 
has no great influence on the results. A-1, B-1 and 
C-1 were given before any other test, and D-1 and E-1 
were given after one of the other three, both volumes 
of the series, had already been administered. In spite 
of this fact, B-1 and C-1 are pronounced the two 
easiest, and A-1, the hardest to read. This would 
seem to indicate that the practice gained from previous 
experience in the test had no great influence on the 
results. 



74 TEXTBOOK SELECTION 

Some schools were chosen in which to change the 
order of the giving of the tests. In considering these 
results we must bear in mind that the determination 
for a single school^ is very unreliable, and if the 
general tendency of the results is like the results of the 
total, in spite of the change in order of administration, 
our conclusions will be verified. In No. 1 School 
the order of giving was E-1, D-1, C-1, B-1, — with the 
following results : 

Per Cent of Correct 
Text Responses 

B-1 .76 

E-1 .67 

C-1 .66 

D-1 .63 

There were seventeen 12-year-olds. 

In School No. 2 the order of giving was E-1, E-2, 
D-2, C-2, — with the following results : 





Per Cent op Correct 


Text 


Responses 


E-2 


.75 


E-1 


.70 


D-2 


.69 


C-2 


.57 



There were one hundred and fourteen 12-year-olds. 

1 The data used here were gathered in a middle western 
city. 



TEXTBOOK SELECTION 75 

In School No. 3 the order of giving was D-1, E-1, 
C-2, A-2, with the following results : 

Per Cent of Correct 
Text Responses 

E-1 .81 

D-1 .75 

A-2 .67 

C-2 .66 

There were forty-six 12-year-olds. 

In School No. 4, the order of giving was D-1, D-2, 
E-1, C-2, — with the following results: 

Per Cent of Correct 
Text Responses 

E-1 .67 

D-1 .65 

D-2 .57 

C-2 .54 

There were sixty-eight 12-year-olds. 

In School No. 5, the order of giving was E-1, E-2, 
C-2, D-2, with the following results : 

Per Cent of Correct 
Text Responses 

E-1 .59 

D-2 .58 

E-2 .57 

C-2 .51 

There were fifteen 12-year-olds. 



76 TEXTBOOK SELECTION 

In School No. 6, the order of giving was D-2, E-1, 
E-2, C-2,with the following results: 





Per Cent of Correct 


Text 


Responses 


E-2 


.39 


E-1 


.37 


C-2 


.27 


D-2 


.22 



There were nine 12-year-olds. 

Certain noticeable features deny ''the order of 
giving" any part in the results. Certainly the test 
which is given last is not always the easiest. The only 
case in which it is easiest is in School No. 1, and there 
it is because B-1 was given last, and B-1 is by our total 
result the easiest. Also there is no case in which the 
test which is given first is demonstrated to be hardest. 
E-1 is always about as hard as D-1, as it is in our totals, 
no matter what the order of giving. As a matter of 
fact, E-1 was easier for the children of School No. 3 
School No. 4 and School No. 5 than D-1, even though 
E-1 was given first in No. 4 and No. 5, and D-1 was 
given first in School No. 3. A-2 bears the position in 
difficulty in School No. 3 that it has in our totals, even 
though it was given last. C-2 is the hardest for the 
children at No. 1, No. 3, No. 4 and No. 5, and next to 
the hardest for the children of No. 6, even though it 
was given last, or next to the last in each, — that is to 



TEXTBOOK SELECTION 77 

say, C-2 which is hardest by our summaries when 
practice in only one test (C-1) had been given, is 
also hardest in these special schools when practice in 
two or three tests had been given before the adminis- 
traction of C-2. E-2 is easier than C-2 and D-2 in No. 
2 and No. 6 even though it is given before C-2 
and D-2, and is about as hard as D-2 and easier than 
C-2 in School No. 5, even though it is given before C-2 
and D-2. 

This analysis of the schools where a special order of 
giving was instituted seems to indicate that the data 
are independent of the order in which the tests were 
given. 

How Reading Difficulty of the Texts Was 
Obtained 

The reports came to us from each school, listing 
the number of correct responses, the number of 
incorrect responses, and the number not attempted, 
for each question. We were therefore able to deter- 
mine the percentage that the correct responses were 
of the total attempts. As indicated above, we threw 
out of consideration any school in which the ''not 
attempted" was larger for any questions at the end 
than for questions at the beginning of the test, since 
this indicated that insufficient time had been given 
to adequately measure the responses of the children 



78 TEXTBOOK SELECTION 

to the material itself. This '^ percentage correct'' 
for each book constitutes the third column in Table 1. 
We have here a picture of the relative ease with which 
twelve-year-olds read the ten volumes in question. 
Table 4 gives these percentages by series. 

As it is somewhat difficult to interpret the meaning 
of these differences, we have translated them into a 
more readily understood medium. Table 5 shows 
the deviations in terms of Standard Deviation and 
the average age necessary to read the texts. The 

percentage of correctness was translated into ^*tY» 

deviation in terms of the variability of twelve-year- 
olds. This deviation was translated into a scale 
ranging from five standard deviations below the mean 
to five standard deviations above the mean, — calling 
the former zero, and the latter 100. This places the 
average at 50. We have values now for each book 
which correspond to McCalFs T-score on the Thorn- 
dike-McCall Reading Scale. In Table 5, B-1 is as 
hard as 43 on the Thorndike-McCall Reading Scale. 
C-1 is as hard as 45 on the Thorndike-McCall Reading 
Scale, etc. 

Whereas these terms are indispensable statistically 
to arrive at a final simplified report, they are in them- 
selves so intricate that it is very important to present 
our relative values in a manner more easily under- 
stood. Once arrived at a T-score, it is a simple 



TEXTBOOK SELECTION 79 

matter to use the Thorndike-McCall age norms to 
find what age is necessary to comprehend any one 
book. B-1 is as hard as 43 on the reading scale, and 
the average age of the T-score of 43 is, according to the 
McCall norms, 130 months. We can therefore, say 
that children must be this old in order to understand 
B-1. So with the other reading ages in Table 5. 
It is also of value to us to know which is the lowest 
grade able to read each book. These values are 
derived through the grade norms published by McCall 
for T-score of his test. We now have before us in 
Table 5 the average age necessary to read each book 
and the average grade necessary to read each book. 

Selecting Texts to Meet the Needs or Particu- 
lar Grades and Students 

One other immediate advantage, other than the 
selection of texts, should be parenthetically answered 
here. By test the mental age of grades used in this 
study was determined. Thus we could see which 
grades in any one school are like a fourth grade normally 
would be; which grades are like a fifth grade normally 
would be, etc. In this way individual needs of each 
grade of this city's schools can be seen at a glance. 
As these scores are on a verbal intelligence test, the 
differences are in exactly the same ability upon which 
the relations between texts are based. The correla- 



80 TEXTBOOK SELECTION 

tion between the Thorndike-McCall and the Wylie 
is high. 

A-1 might very well be the best book in every other 
respect than comprehension. In fact, its very values 
may be associated with a necessary sacrifice in terms 
of comprehension. In that case it might be very 
advisable to use A-1 in all schools where they are 
able to comprehend it, and to use the book which 
may not be as valuable, but easier to read, only in 
those schools where it is necessary to sacrifice the 
other values in favor of the sine qua non, comprehen- 
sion. The statement that A-1 is better in other ways 
is of course entirely hypothetical. 

The Vaeiability of Reading Difficulty Within 
A Single Text 

There is a further characteristic of geography text 
books which our data will allow us to analyze. The 
difficulty of a book as a book is one major considera- 
tion in the treatment of comprehension value, but 
the degree to which the difficulty varies from very 
easy to very hard is a very important supplementary 
consideration. B-1 is easier than any of the other 
volumes-1 to read. However, if this average ease of 
comprehension were the result of the product moment 
of very easy and very hard passages, it would never- 
theless be an undesirable text. We want a book the 
difficulty of which is reasonably constant throughout. 



TEXTBOOK SELECTION 81 

Tables 6 and 7 present the T-scores and reading 
ages of each question. The reading age is to be 
interpreted as the average age necessary to read the 
passage and answer the questions as per the conditions 
of our test. Some of the disparity in ages is of course 
due to the sampHng of passages and the chance diffi- 
culty of the questions, but the range of difficulty of 
the text book is certainly pictured in a rough manner 
by the range in difficulties of these passages. 

It is especially of note that the difficulty of the 
sentences in D-1 and D-2 marches progressively from 
easy to hard, which fact is already noted in connection 
with Table 1, where the first halves of these two books 
were shown to be easier than the second halves. The 
questions are in serial order, and samplings are from 
the beginning to the end of the book. Whether or not 
this feature is desirable is dependent upon the progress 
which children can make in ability to read. The 
difference in difficulty between the end and beginning 
of the D's is not justified by the progress which 
children make in the time covered by the books. 

Table 8 gives the standard deviation of reading 
ages of the questions of each book and the average 
reading age of the book. It is because of dropping of 
decimals that the average age of questions per book 
does not check with the age necessary to read the 
book, — computed directly. However, the order of the 
books is of course the same. A consideration of these 



82 TEXTBOOK SELECTION 

measures of variability of the difficulty of passages 
from the various texts expressed in terms of age 
shows that as a tendency, the first and second volumes 
of a series have variabilities very much alike. Note 
that B-2 and B-1 are together; note that D-1 and D-2 
are together; note that A-1 and A-2 are very close 
together ; note that E-1 and E-2 are fairly close together. 
C-2 and C-1 are very different in variability, but all the 
other series have Standard Deviationsof the volumes-1 
which show a likeness to the Standard Deviations of 
volumes-2. The highest Standard Deviations are for 
A-2, C-2 and A-1. These three volumes are among 
the four most difficult to read (Table 5). The lowest 
standard deviations are those for C-1, E-2, B-2 and B-1. 
All of these, except E-2 are among the five books which 
are easiest to read (Table 5). 

There is then, luckily, an association of comprehen- 
sion virtues in these texts. Those that are hard to 
read are also spotty in their difficulty and those that 
are easy to read have an even tenor of difficulty. 

The Success of the Texts in Reaching Their 
Reading Public 

Table 9 gives the frequency of each reading age 
of all questions on all volumes-1. An analysis of 
this frequency distribution of average age necessary 
to answer each of 104 questions, after reading the 



TEXTBOOK SELECTION 83 

passages selected from five geography text books 
meant for beginners in geography in grades 4 and 5, 
will allow us to judge how far the authors of the texts 
have reached their public. Their public is, of course, 
the children themselves, — not what teachers think of 
those children. Although the sales today may be 
largely based on what superintendents and teachers 
think children of the fourth and fifth grade can read, 
rather than what they actually do read, it is just such 
data as these which will change that emphasis and 
allow us to judge text books upon a basis of their real 
ability to get across to the children that they are 
meant for. The average of the average age necessary 
to read these questions is 135.79 months. With a 
standard deviation of about a year and a quarter, 
about five-sixths of the material is too hard for children 
in grades 4 to understand. Since one standard 
deviation below the mean of the distribution, repre- 
sented in Table 9, is at about 120 months, if we count 
the children in grades 4 as nearly all younger than 120 
months in reading ability, five-sixths of the passages 
of the texts given them are too hard for the group. 
Volumes-2 are a little more adequate to the task 
which they pretend to perform. The average of the 
average age necessary to read the 111 questions on 
volumes-2 is 143.50 months. The standard devia- 
tion of reading age of questions is again about a year 
and a quarter. This distribution of reading ages by 



84 TEXTBOOK SELECTION 

questions on volumes-2 will be found in Table 9. If 
volumes-2 are meant primarily for grades 6 and 7, 
only about one-half can be understood by the children 
of the grade in which they are first introduced, (Grade 
6.) Since the average age necessary is about twelve 
years, if we count most sixth-graders as not more than 
twelve years old in reading ability they are then able 
to understand only one-half of the matter presented 
to them in all geography texts meant for the grade. 
This one-half is not the first halves of the book as is 
apparent from Table 7. 

Individual Differences in Needs of Various 
Grades 

On January 14, 1921, the 5-A grades of the schools 
used in this research, about to enter on their 6th 
grade work had average scores on the Wylie A, B and 
C ranging from 55.15 to 19.69. Interpreted, this 
means they range from an average age of 12 years 
6 months to 8 years 6 months, in the ability measured 
by the Wylie test. They differ in the ability of work 
which they can do as measured by the Wylie test 
from average 7th grade capacity at beginning of B 
work to average 4th grade capacity (at point of 
promotion from B to A). 

Associating these facts with those of Table 5, it is 
apparent that some 6th grades are able to use any of 
the texts, whereas others need to choose the very 



TEXTBOOK SELECTION 85 

easiest and then still will have a book which is too 
difficult. The best 6th grades can just read the hardest 
volume-2 and not all can read the easiest. This is a 
very valuable practical consideration since we may be 
able to determine as suggested above that a certain 
book excels the others in all qualities save ease of 
reading. Then we must know in which grades the 
difficulty of comprehension makes no difference in 
order to benefit where we can by the excellence of the 
book in other qualities. 

In order to find which 6th grades (and for volumes-1 
which 4th grades) are able to use a difficult book, 
consult the mental age ratings of each grade. 

The Text Already in Use Had Negligible 
Advantage 

In conclusion a word regarding the geography now 
in use in the lower grades is necessary to assure us that 
training in it has not prejudiced our results in favor 
of one of the books. *'B," First Book, Part One has 
been used the last three years. Since the first 90 
pages of the 256 (exclusive of indices) of our B-1 is 
this same Part One, we must be sure that questions 
on this portion are at least as hard as questions on the 
remainder of B-1. This portion that has been studied 
covers questions on our test 1 to 5 inclusive. Though 
three of these five questions were comparatively easy 
for our 12-year olds (see Table 6) still Table 1 shows 



86 



TEXTBOOK SELECTION 



that the second half of B-1 (which was not studied) 
was easier than the first half (part of which was 
studied). We feel therefore that no important advan- 
tage was given the "B'^ series. 

This treatise is of comprehension value of the texts 
only. It makes no claims to ascertain any of the 
other necessary virtues of a text. When those too are 
credited through objective means we will be able to 
choose texts, not risk them. 



Table 1 





Percentage of 


Percentage of 


Percentage of 




questions on 


questions on 


questions on 


Key number 
of text 


first half of 


second half of 


total of test 


test answered 


test answered 


answered cor- 


correctly by a 


correctly by a 


rectly by a 




normal group 


normal group 


normal group 




of 12-year-olds 


of 12-year-olds 


of 12-year-olds 


B-1 


68 


77 


72 


C-1 


71 


67 


69 


D-1 


79 


64 


66 


E-1 


64 


68 


66 


B-2 


63 


64 


64 


E-2 


62 


65 


64 


A-1 


62 


61 


62 


D-2 


64 


51 


58 


A-2 


52 


59 


56 


C-2 


54 


48 


51 



TEXTBOOK SELECTION 

Table 2. — Distributions in Wylie A, B and 

January 14, 1921 by Groups of Schools 

Taking A, B, C, D and E Geography 

Reading Tests-Eleven-year-olds in 

Those Schools at That Time 



87 
C, 





A 


B 


C 


D 


E 







19 


34 


31 


41 


33 




0- 5 


11 


29 


24 


31 


35 


— »10 % of the number below this 


5- 10 


22 


21 


18 


35 


25 


Una in each distribution. 


10- 15 


6 


9 


4 


10 


12 




15- 20 


11 


7 


14 


18 


18 




20- 25 


9 


15 


17 


18 


22 


— »25 % of the number below this 


25- 30 


12 


24 


17 


29 


27 


Une in each distribution. 


30- 35 


20 


15 


26 


35 


31 




35- 40 


20 


18 


21 


27 


31 




40- 45 


11 


23 


25 


24 


32 




45- 50 


20 


36 


27 


36 


36 


— »This line halves the total 


50- 55 


26 


47 


37 


46 


56 


number in each distribution. 


55- 60 


28 


35 


32 


48 


47 




60- 65 


37 


51 


35 


64 


46 


—^25 % of the number above this 


65- 70 


26 


44 


29 


40 


38 


line in each distribution. 


70- 75 


17 


23 


28 


37 


24 


-^10 % of the'number above this 


75- 80 


15 


14 


17 


17 


17 


line in each distribution. 


80- 85 


13 


13 


11 


12 


17 




85- 90 


8 


6 


8 


10 


7 




90- 95 


5 


7 


9 


12 


7 




95-100 


4 


7 


8 


9 


6 




100-105 


2 


4 


4 


6 


3 




105-110 





1 


2 


3 







110-115 


1 


1 


2 


3 







115-120 


1 


2 


1 


1 


2 




120-125 






1 




1 






344 


486 


448 


612 


573 





88 



TEXTBOOK SELECTION 



Table 3. — Distributions in Wylie A, B, and C, 

January 14, 1921, by Groups of Schools 

Taking A, B, C, D and E Geography 

Reading Tests — Twelve-year-olds in 

Those Schools at That Time 





A 


B 


C 


D 


E 





18 


29 


17 


28 


35 




0- 5 


11 


10 


17 


27 


17 




5- 10 


11 


9 


13 


23 


12 




10- 15 


4 


7 


6 


12 


8 




15- 20 


6 


7 


10 


16 


14 




20- 25 


3 


12 


11 


16 


9 




25- 30 


7 


11 


12 


17 


16 




30- 35 


14 


22 


10 


23 


23 




35- 40 


14 


14 


23 


25 


28 




40- 45 


26 


23 


12 


34 


23 




45- 50 


19 


19 


24 


37 


26 




50- 55 


37 


30 


39 


51 


56 




55- 60 


37 


28 


34 


46 


40 




60- 65 


35 


39 


37 


58 


60 




65- 70 


30 


40 


35 


50 


45 




70- 75 


21 


23 


33 


33 


34 




75- 80 


16 


21 


26 


31 


35 




80- 85 


20 


18 


25 


32 


23 




85- 90 


13 


11 


18 


15 


19 




90- 95 


9 


5 


13 


13 


9 




95-100 


8 


2 


7 


3 


8 




100-105 


2 


4 


8 


5 


6 




105-110 





4 


4 


8 


3 




110-115 


1 


4 


4 


4 


4 




115-120 


1 





3 


1 







120-125 




2 


1 


3 


1 




125-130 




2 


1 





2 




130-135 






397 


1 

444 



612 


556 






363 





TEXTBOOK SELECTION 89 





Table 4 






Percentage op Questions 


Series 




Correct 


B 




67.7 


E 




64.8 


D 




62.1 


C 




59.4 


A 




58.6 



90 



TEXTBOOK SELECTION 



Table 5. — Distinctions in Compeehension Value 
Translated into Age and Grade 





Proportion 


X 

„ _^ in a 
S. D. 


T-scale 
like Thorn- 


Average 


Average 
grade 


Text 


of correct 
responses 


normal dis- 
tribution of 


dike-Mc- 
Call read- 


sary to com- 
prehend 


necessary 
to com- 






difficulty 


ing scale 


prehend 


B-1 


.72 


-.58 


44.2 


133 


5.48 


C-1 


.69 


-.50 


45.0 


135 


5.50 


D-1 


.66 


-.41 


45.9 


138 


5.66 


E-1 


.66 


-.41 


45.9 


138 


5.66 


B-2 


.64 


-.36 


46.4 


139 


5.74 


E-2 


.64 


-.36 


46.4 


139 


5.74 


A-1 


.62 


-.31 


46.9 


141 


5.82 


D-2 


.58 


-.20 


48.0 


144 


6.00 


A-2 


.66 


-.45 


48.5 


145 


6.09 


C-2 


.51 


-.02 


49.8 


150 


6.31 



Whole 








Note columns above for 


series 








age and grade per volume 


B 


.68 


-.47 


45.3 


135 


5.51 


E 


.65 


-.39 


46.1 


138 


5.69 


D 


.62 


-.31 


46.9 


141 


5.82 


C 


.59 


-.23 


47.7 


143 


5.95 


A 


.59 


-.23 


47.7 


143 


5.95 



TEXTBOOK SELECTION 



91 



Table 6. — T-scores and Reading Ages by Ques- 
tions IN Tests on Volumes-1 



Number of 
fquestion 


g 


bfi 


g 


bO 

a 




.S 


1 


bO 


o 






H 


Ph 


H 


Ph 


y 


^ "^ 


H 


f^ "^ 


H 


« ^ 




B-1 


C-1 


D-1 


E-1 


A-1 


1 


40.1 


121 


48.2 


145 


39.2 


119 


34.4 


105 


42v6 


129 


2 


35.2 


108 


46.7 


140 


39.2 


119 


41.2 


125 


40.8 


123 


13 


45.3 


136 


38.2 


117 


33.5 


102 


52.8 


157 


50.3 


151 


4 


46.7 


140 


43.3 


131 


39.6 


120 


44.5 


135 


35.9 


110 


5 


35.9 


110 


42.9 


130 


40.5 


123 


46.4 


139 


47.7 


143 


6 


45.0 


135 


43.6 


132 


43.6 


132 


37.7 


115 


44.5 


134 


7 


49.5 


148 


51.0 


152 


38.7 


118 


54.7 


163 


46.1 


138 


8 


46.1 


138 


42.6 


129 


46.9 


141 


51.8 


154 


48.2 


145 


9 


56.7 


169 


40.5 


122 


43.6 


132 


46.7 


140 


54.1 


161 


10 


42.3 


128 


40.8 


123 


44.5 


134 


45.9 


138 


61.3 


183 


11 


36.6 


112 


45.3 


136 


42.9 


130 


37.2 


114 


42.6 


129 


12 


45.3 


136 


42.9 


130 


45.3 


136 


49.5 


148 


42.9 


130 


13 


49.2 


148 


45.3 


136 


48.2 


145 


40.8 


123 


46.9 


141 


14 


38.2 


117 


44.2 


134 


54.1 


161 


46.9 


141 


46.7 


140 


15 


39.2 


119 


51.0 


152 


45.6 


137 


45.9 


138 


48.7 


146 


16 


42.6 


129 


43.3 


131 


38.7 


118 


48.0 


144 


54.7 


163 


17 


44.5 


135 


42.3 


128 


45.6 


137 


42.6 


129 


47.5 


143 


18 


42.6 


129 


48.5 


146 


47.5 


143 


49.5 


149 


56.7 


169 


19 


42.9 


130 


47.7 


143 


56.7 


169 


44.2 


134 


52.0 


155 


20 










39.2 


119 






36.6 


112 


21 










51.5 


153 






42.0 


127 


22 










56.1 


167 






41.6 


126 


23 










50.8 


152 






42.9 


130 


24 










50.5 


151 











92 



TEXTBOOK SELECTION 



Table 7. — T-scores and Reading Ages by Ques- 
tions IN Tests on Volumes-2 



Number of 
question 




ii 


i 


1. 


o 
o 


.S 

1 s 




1i 


0) 






H 


rt ^ 


H 


rt ^ 


H 


jS C3 


H 


tf * 


H 


« * 




B-2 


E-2 


D-2 


A-2 


C-2 


1 


40.1 


121 


52,0 


155 


45.3 


136 


57.1 


169 


52.8 


157 


2 


53.3 


159 


47.5 


142 


41.6 


126 


55.5 


166 


44.8 


135 


3 


45.0 


135 


46.9 


141 


45.9 


138 


59.2 


176 


46.9 


141 


4 


47.5 


143 


44.8 


135 


45.9 


138 


58.0 


172 


42.0 


127 


5 


47.7 


144 


42.6 


129 


51.3 


153 


58.4 


173 


46.4 


139 


6 


47.7 


144 


51.0 


152 


53.9 


161 


59.2 


176 


55.8 


166 


7 


50.8 


152 


41.6 


126 


45.3 


136 


46.1 


138 


54.7 


163 


8 


45.3 


136 


41.2 


125 


37.7 


115 


49.2 


148 


56.1 


167 


9 


46.1 


138 


53.3 


159 


40.8 


123 


41.2 


125 


37.2 


114 


10 


42.3 


128 


42.0 


127 


43.9 


133 


49.0 


147 


46.7 


140 


11 


44.2 


134 


40.5 


122 


47.7 


143 


42.0 


127 


52.5 


156 


12 


49.7 


149 


51.8 


154 


51.8 


154 


42.0 


127 


57.7 


171 


13 


52.0 


155 


52.3 


156 


47.2 


142 


35.9 


110 


57.1 


169 


14 


45.9 


138 


44.5 


135 


53.1 


158 


41.6 


126 


50.8 


152 


15 


54.4 


162 


43.9 


133 


53.6 


160 


41.2 


125 


47.2 


142 


16 


38.2 


117 


43.9 


133 


47.5 


142 


57.1 


169 


36.6 


112 


17 


37.7 


115 


45.6 


137 


51.0 


152 


49.5 


148 


55.8 


166 


18 


45.3 


136 


47.7 


143 


59.5 


177 


51.0 


152 


51.5 


154 


19 


46.7 


140 






43.6 


132 


56.1 


167 


38.7 


118 


20 


43.6 


132 






45.9 


138 


48.7 


146 


50.0 


150 


21 


45.9 


138 






42.9 


130 


45.3 


136 


53.9 


161 


22 


.... 








43.6 


132 


47.7 


143 


53.3 


159 


23 










51.5 


153 


42.6 


129 






24 










60.4 


179 


48.0 


144 






25 














45.3 


136 






26 




... 




... 


.... 




40.5 


123 







TEXTBOOK SELECTION 



93 



Table 8. — Standard Deviations and Averages of 

Reading Ages of the Questions by Book 

IN Order of Size of Standard Deviations 



Book 


S.D. 


M 


C-1 


9.45 


134.58 


E-2 


11.48 


139.11 


B-2 


11.79 


138.86 


B-1 


14.55 


130.95 


E-1 


14.64 


136.37 


D-2 


15.56 


143.79 


D-1 


16.79 


135.75 


A-1 


17.32 


140.35 


C-2 


17.66 


148.14 


A-2 


18.79 


146.08 



94 



TEXTBOOK SELECTION 



Table 9. — Distkibution of Reading Age Neces- 
sary TO Read Questions on Volumes-1 and 

VOLUMES-2 



Reading age 


Volumes-1 


Volumes-2 


100-105 


1 




105-110 


2 




110-115 


5 


3 


115-120 


9 


4 


120-125 


7 


4 


125-130 


11 


13 


130-135 


15 


8 


135-140 


15 


19 


140-145 


12 


15 


145-150 


9 


|5 


150-155 


8 


11 


155-160 


1 


9 


160-165 


4 


5 


165-170 


4 


8 


170-175 


, . 


3 


175-180 


. . 


4 


180-185 


1 






Volumes-1 


Volumes-2 


N 


104 


111 


M 


136.20 


143.95 


S.D. 


15.25 


16.05 



LIBRARY OF CONGRESS 



021 286 953 3 



